Dimensionality Reduction by Random Mapping: Fast Similarity Computation for Clustering

When the data vectors are high dimensional it is com putationally infeasible to use data analysis or pattern recognition algorithms which repeatedly compute simi larities or distances in the original data space It is therefore necessary to reduce the dimensionality before for example clustering the data If the dimensionality is very high like in the WEBSOM method which orga nizes textual document collections on a Self Organizing Map then even the commonly used dimensionality re duction methods like the principal component analysis may be too costly It will be demonstrated that the document classi cation accuracy obtained after the di mensionality has been reduced using a random mapping method will be almost as good as the original accuracy if the nal dimensionality is su ciently large about out of In fact it can be shown that the inner product similarity between the mapped vectors follows closely the inner product of the original vectors